This invention pertains to the storage and interactive playing of segmented continuous play media sequences and more particularly to digitally stored interactive multimedia content.
Continuous play media (“multimedia”) can be defined as minimally containing a stream of image or text content forming a perceived continuity when presented to an observer/user. Continuous play media may further contain content streams forming perceived audio continuities when presented to an observer/user. Continuous play media may further contain other content streams including but not limited to closed captioned support for the acoustically impaired or closed captioned presentation of dialogue in one or more alternative languages.
Image content streams as used herein will include but not be limited to motion video streams as well as streams of text moving with regards to the displayed view available to the user/observer. Programs may generate these image streams, where the displayed view is altered in a manner rendered essentially continuous. Implementation environments for such generating programs include but are not limited to language environments for C, C++, Java, Lingo™ by Macromedia as well as various low level machine independent and/or machine dependent assembly languages. Examples of such implementations include but are not limited, to motion picture preambles and postambles, often incorporating still images, textual credits and copyrights often further accompanied by music with a motion background or moving text providing a sense of continuity to the user/observer. Such image content streams may be further accompanied by acoustic effects which augment, and in some cases, establish the sense of continuity experienced by the user/observer.
Contemporary technology has driven down the cost of storing continuous play media, allowing the recording of large amounts of audio-video content onto a compact disk or DVD disk by a growing number of people, businesses and institutions. Continuous play media recordings and the playing of such recordings began in large numbers with the video tape players and their successors, Video Cassette Recorders (VCRs). Such technologies could record and play one to several hours of television grade audio-video content. The user could control the player to rewind, fast forward, pause, stop and start at a given moment in the recorded presentation. Many of these units could record material, usually received from a television signal, as well as play pre-recorded material. Standard implementations of this technology record or play only one multi-media stream at a time without multiplexing schemes supporting multiple simultaneous streams.
More recently, digital continuous play media technologies based upon image compression techniques such as QuickTime™ by Apple, MPEG 1, MPEG 2 and DVD have become quite popular. Such technologies are typically used to create a single long playing sequence, such as a motion picture, documentary or training session. User controls for playing such recordings are very similar to earlier controls found on video tape players and VCRs. All of these prior methods of playing and storing continuous play media have a limitation in presenting complex subjects, namely that there presentation is flat, there is no way for the user/observer to alter the stream to delve into a topic as a contiguous part of the stream.
The Internet and World Wide Web have accelerated the proliferation of hypertext documents. Hypertext possesses highlighted triggers embedded into a viewed document, which when selected, cause the document viewer to display a different view of either the same document or a different document. There is a “back” button on most hypertext viewing systems, which when pushed, causes the view to return to the previous view. These hypertext documents are rapidly creating a new class of interactive documents, allowing a much higher level of complexity to be traversed by readers of greatly varying backgrounds. People may follow many of the hypertext links, or few of the hypertext links depending upon their preferences. However, the approach of hypertext is not continuous play media, it does not create audio-visual streams, but screens of text and pictures primarily, which only move when the user/observer urges them to move. When a hypertext document references a continuous play media file, such as a QuickTime file, it is loaded and played with controls much as a VCR possesses.
Television, long one of the dominant cultural forces in image content presentation has reached a difficult impasse. Television in the United States is largely supported by revenues from advertising. The Internet threatens such revenues. The reason is that many perceive the Internet as a better basis for advertising because people who are interested in a product can find out what they want to know about the product as they wish to find it out via hyper links. Today's television advertising is constrained to present sound bites of very short duration possessing no ability for the potential customer to direct an inquiry into facts they wish to know. Instead, television today forces repeated transmission of the same limited amount of information, never getting beyond the simplest of messages concerning a product. What is needed is a method by which an advertiser's purchased bandwidth can be more efficiently utilized to permit potential customers to query a larger cross section of information about the products advertised while insuring that the basic product pitch is seen.
FIG. 1 displays a relevant prior art system comprising an enclosure 10, housing a display device 12, selector device 14, and communication (16 and 18) between selector device and system plus speakers (20). Enclosure 10 is shown herein with minimal detail by way of illustration. In practice, prior art system enclosures relevant to this invention include but are not limited to television-style cases, desktop computer enclosures, notebook computer enclosures. Many of these enclosures 10 incorporate speakers 20 without them being perceived separately as indicated in this figure. Note that there are a number of systems containing more than one enclosure, such as a number of desktop computers, televisions with set top boxes and often, additional content players such as DVD players.
Relevant prior art display devices 12 are also widely varied in form and specifics of operation. Relevant prior art display devices 12 may present black and white or color images. Relevant prior art display devices 12 may support either a vector or raster format. Relevant prior art display devices 12 may present images in either a 2-D, 3-D or multi-dimensional presentation view or collection of views.
Relevant embodiments of selector device 14 include but are not limited to contemporary television channel selectors, home entertainment center remote controls, computer pointing devices including but not limited to 3-D and 2-D mouse style pointers, pen tablets, track balls, touch pads, key pads and joysticks. As illustrated in FIG. 1, the selector device communicates via physical transport mechanism 16 with an interface 18 housed in enclosure 10. Relevant physical transport mechanisms 16 include but are not limited to infra-red, micro-wave and other similar wireless transport layers, as well as wires and optical fiber. The mechanism by which communication is carried out based upon the specific physical transport mechanism employed is not relevant to this invention and will not be discussed for that reason. Additional IO devices such as printers and keyboards may be attached to various relevant, prior art systems. Keyboards may house touch pads and mouse sticks which in certain cases are the relevant selector device of that system.
FIG. 2 displays a block diagram of an exemplary prior art system such as displayed in FIG. 1. The units (12, 14, 20, 36 and 44) on the left side of this figure all have a major role in the input and output flows processed and controlled by the second column of units (24, 18, 32, 40 and 48), respectively. The data transport mechanisms between units (12, 14, 20, 36 and 44) and units (24, 18, 32, 40 and 48) are represented by arrows (22, 16, 30, 38 and 46), respectively. These units interact with each other and an overall control circuit labeled digital processor 56 via arrows representing buses (26, 28, 34, 42, 50, 52 and 54). Digital processor 56 in turn has RAM 62 and Nonvolatile memory 66 which it controls and uses to direct the overall operation of relevant prior art systems via buses designated as arrows (58, 60, 64 and 68).
Relevant prior art display devices 12 may present black and white or color images in either a vector or raster format representing images in either a 2-D, 3-D or multi-dimensional presentation view or collection of views. Relevant display data transport 22 includes but is not limited to NTSC, PAL or various HDTV television protocols of either analog or digital formats, as well as digital and analog RGB and various flat panel display interface protocols as are often used with computer displays. Many systems today possess a specialized display controller 24, which often incorporates one or more temporary frame buffers and MPEG decoding acceleration technology as well as acceleration technology for a variety of graphics operation. The communication mechanism 26 by which these units interact with the rest of an exemplary prior art system include but are not limited to microcomputer busses such as PCI and AGP as well as dedicated communication paths represented within line 52.
The selector device 14, selector device communication mechanism 16 and selector interface 18 have been discussed above. The communication between the selector interface 18 and the rest of the system is denoted by arrow 28. Embodiments of arrow 28 include but are not limited to addressable interfaces on computer busses including but not limited to ISA, PCI and USB.
Relevant prior art speakers 20 communicate with audio generator 32 via arrow 30. Arrow 30 designates communication mechanisms including but not limited to analog signing or digital signaling. Arrow 30 communications may be physically transported by wireline technology including but not limited to twisted wire pairs and coaxial cabling, as well as wireless technology including but not limited to short range radio and infra red mechanisms. Audio generator 32 may perform decoding functions such as translation of encoded MPEG audio streams delivered by arrow 34 from the overall internal communications network 52 possibly from the player controller 40 or external interface 48 or display controller 24. Power amplification of the audio signals may be done either by audio generator 32, within the speakers 20 or involve both. Embodiments of the audio generator interface 34 to the internal communications network 52 include but are not limited to addressable interfaces on computer busses including but not limited to ISA, PCI and USB.
Relevant, prior art content player 36 communicates with content controller 40 via arrow 38. Content player 36 embodiments include but are not limited to optical disk players and electromagnetic disk players of either a removable or non-removable media. These content players 36 embodiments further include but are not limited to CD ROM, MPEG and DVD players. Such content player 36 embodiments may further include the ability to write to the storage media as well as play the storage media. Relevant player controller 40 embodiments include but are not limited to various SCSI controllers, specialized optical disk controllers, specialized hard disk controllers and RAID disk array controllers. Player controller 40 embodiments may further include but are hot limited to various continuous play media compression decoders: MPEG decoders and DVD decoders. Relevant prior art communications mechanisms 38 include but are not limited to various SCSI, RAID, ISA and EISA interfaces.
Relevant prior art player controllers 40 often, but not always, partition a continuous play media stream received via 38 from content player 36 into an audio stream and a video stream. The audio stream is sent via communication path (arrows 42 to 52 to 34) to audio generator 32 to drive speakers 20. This communication path may be incorporated into an overall bus protocol, or be a separate signal path, depending upon the specific implementation. The video stream would be sent via communication path (arrows 42 to 52 to 26) to display controller 24 to drive display 12. The actual decoding of the video stream is often done primarily in display controller 24, but in certain instances, digital processor 56 and player controller 40 may contribute to the video decoding process. There are also situations in which relevant prior art systems employ display controller 24 to partition a continuous play media stream into separate one or more audio streams and at least one video stream. Such systems include many set top box architectures.
Note that in relevant prior art systems, there may be more than one content player 36 with potentially distinct player controllers 40 and communication paths 38. One content player 36 might support a writeable CD ROM using a SCSI 38 based controller 40 as well as a second DVD-ROM player with its own cabling 38 and player controller 40.
Another relevant source of continuous play media content is provided via external content 44 communicating with external interface 48 via arrow 46. One relevant external interface 48 is a radio frequency (RF) tuner. Relevant RF timers 48 include but are not limited to demodulators and/or modulators for various broadcast protocols such as Frequency Modulation (FM), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), various spread spectrum protocols, Wavelength Division Multiple Access and wavelet division multiple access. Relevant spread spectrum protocols further include but are not limited to Direct Sequence, Frequency Hopping, Time Hopping and Wideband CDMA. These relevant RF tuners may be connected 46 by wireline or wireless physical transport layers. Relevant wireline physical transports include but are limited to twisted pair, coaxial cable and various optical fiber mechanisms. Relevant wireless physical transports 46 include contemporary broadcast television, High Definition TV (HDTV), as well as various radio frequency, microwave and infra red schemes which may well incorporate an antenna, sensor or array of antennas or sensors.
Another relevant external interface 48 is a modem. Relevant modems include but are not limited to telephone line modems incorporating various transceiver rates which may not be the same for reception as for transmission, as well as various DSL, ADSL, XDSL, ISBN, Ethernet, Token Ring and ATM interfaces. Physical transport layer 46 for modems include but are not limited to wire line and wireless transport layers. Wire line physical transport, layers 46 include but are not limited to telephone lines, twisted pair wirelines coaxial cabling and various optical fiber technologies. Wireless transport layers 46 include but are not limited to directional and non-directional radio, microwave, infrared and optical schemes.
The external content 44 may be located a substantial distance away from the enclosure 10. The external content 44 is often embodied in many circumstances within a server supporting a network of user systems via interconnections 46 of these external interfaces 48. Such networks may well support TCP/IP thereby enabling support for the Internet. Such networks may further support one or more Intranets. Such networks may further support one or more Extranets.
Another form of external content 44 includes video input devices. These often possess external interfaces 48, which include video frame capturing circuitry. Such external interfaces 48 are now including advanced image processing, often further supporting MPEG compatible compression of the captured video stream.
Note that in many relevant prior art systems, there is more than one kind of external content 44 and external interface 48 with potentially different communication paths 46. A desktop box might possess both a RF tuner using an antenna as well as an optical fiber interface to a cable television provider. A notebook computer might well have both a telephone line modem and an Ethernet LAN interface.
Relevant prior art digital processor 56 embodiments include but are not limited to one or more of the following: general purpose microprocessors, Digital Signal Processors (DSPs), parallel processors, embedded controllers and special purpose system controllers. General purpose microprocessors include but are not limited to various word width Complex Instruction Set Computers (CISC) and Reduced Instruction Set Computers (RISC). DSPs include but are not limited to various word width computers employing instruction sets allowing at least one add/subtract operation as well as at least one operation comparable to multiplication to be performed in a single instruction cycle. Parallel processors include but are not limited to Single Instruction Multiple Datapath (SIMD), Multiple Instruction Multiple Datapath (MIMD), and hybrid SIMD/MIMD organizations of either uniform or non-uniform processors. Uniform processor parallel processors employ essentially the same processor uniformly. Non-uniform processor parallel processors do not employ essentially the same processor throughout. Embedded controllers often incorporate either one or more microprocessors or DSPs along with additional circuitry performing specialized data processing, which may include but is not limited to MPEG stream partitioning and/or decoding, copy protection processing, decryption, authentication and block data error detection and correction. Special purpose system controllers include but are not limited to various implementations as Programmable Logic Arrays (PLAs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs) and Application Specific Standard Products (ASSPs).
Relevant prior art digital processors 56 often possess local memory resources in the form of RAM 62 and nonvolatile memory 66, interfaced via busses 58, 60, 64 and 68. The RAM may include but is not limited to various forms of RAM and brie or more caching banks of RAM. Relevant prior art digital processor 56 embodiments may include but are not limited to one or more of memory caches physically proximate to and possibly contained within the digital processor 56 package or packages. Memory caching may include but is not limited to separate caching of memory and data. Memory caching may further include but is not limited to multiple layers of cache structures. Distinct processors within the digital processor 56 may further possess distinct caches as well as further localized memory which may in turn include RAM and/or nonvolatile memory. Relevant prior art nonvolatile memory may include but is not limited to boot ROMs and flash memory circuits which may further emulate disk drives with a form of file management system. Such nonvolatile memory 66 embodiments may be used to initialize the system as well as provide security and accounting information or store content.
From the user perspective, relevant prior art systems play continuous play media content recordings much as did predecessor VCR systems. They can start at the beginning, or partially through a sequence, and progress forward pausing, stopping and possibly rewinding. Internet access has allowed hypertext-based web sites to provide hot keys supporting the downloading and playing of continuous play media sequences, but again, once downloaded, the continuous play media sequence playing controls are essentially those found on a VCR. Modern television, particularly cable and satellite broadcast television possesses a fairly large number of channels, often over a hundred, to be multiplexed and modulated at the broadcast site and demodulated at the customer/user site into these separate channels. The signal protocols in common use deserve some discussion. These relevant, prior art television channels tend to employ an MPEG continuous play media stream, with an audio stream and a video stream component.
Consider first what happens in the video stream. MPEG video compression mechanisms utilize an initializing video frame compressed in a manner similar to a still frame, followed by motion compensation data essentially transforming this initial frame into a succession of subsequent motion frames. Every so often, a new initializing frame is sent, followed by motion compensation data again transforming the new initial frame into a new sequence of subsequent motion frames. The compression ratios compared to the raw data are impressive, often a reduction in data size of 200 to 1 can be achieved without noticeable loss of visual clarity. This compression ratio enables this large number of channels to be cost effectively broadcast by satellite and cable television companies.
Consider what happens when someone changes television channels. The MPEG stream of the new channel is isolated and demodulated from the television broadband transmission into a video stream and an audio stream. The video stream is scanned until the next initializing video frame is encountered. Once encountered, the MPEG video decoder initializes its output stream and motion video frames follow shortly from this initialization frame. These initialization frames occur frequently enough that there is a barely noticeable delay between when one turns to a new channel and the channel's video stream is being displayed. If at this point, the channel is again changed, a short time later there is another initialization video frame observed, the motion frame sequencing begins again. However, there is no mechanism to compensate for or retain whatever has transpired on the original channel. Once these motion frame sequences have passed by, they are gone.
The audio streams also possess an initialization structure which is asserted from time to time, followed by the time varying incremental audio stream modifications. While the audio and video frames do not necessarily initialize simultaneously, there are synchronization controls between them to facilitate time-aligning them to each other. Taken collectively, there is a certain rationale in considering contemporary continuous play media to be segmented. The block coding used in both transmission and storage of digital continuous play media reinforces the sense of segmentation of continuous play media into short sequences of motion frames (video stream) and the associated audio stream. However, there are no embedded cues within these streams to indicate expansion possibilities or to direct such expansions or direct the return to provide continuity with the original stream segments.
What is needed is a method of playing and storing a segmented continuous play media stream which expands and contracts the viewing material based upon the user/observer's and/or system/agent(s)' selections.